System and method for identifying a person with reference to a sclera image

ABSTRACT

A method for obtaining an identification characteristic for a subject includes acquiring an image of an eye of the subject, segmenting the eye image into different regions, extracting features in a sclera region segmented from the eye image, and generating data identifying at least one feature extracted from the sclera region of the eye image.

GOVERNMENT INTEREST

This invention was made with government support under contractN00014-07-1-0788 awarded by the Office of Naval Research. The governmenthas certain rights in the invention.

PRIORITY CLAIM

This application claims priority from International ApplicationPCT/US10/20991, which is entitled “SYSTEM AND METHOD FOR IDENTIFYING APERSON WITH REFERENCE TO A SCLERA IMAGE,” and was filed on Jan. 14,2010. This application claims further priority from U.S. ProvisionalApplication No. 61/144,508 filed on Jan. 14, 2009 and from U.S.Provisional Application No. 61/260,451 which was filed on Nov. 12, 2009.

TECHNICAL FIELD

The system and method described below relate to the identification of aperson or an animal with reference to external physical characteristicsof the person or animal, and, more specifically, with reference toexternally observable physical characteristics of one or more eyes ofthe person or animal.

BACKGROUND

Systems for identifying persons through intrinsic human traits have beendeveloped. These systems operate by taking images of a physiologicaltrait of a person and comparing information stored in the image to imagedata corresponding to the imaged trait for a particular person. When theinformation stored in the image has a high degree of correlation to therelevant data previously obtained for a particular person's trait,positive identification of the person may be obtained. These biometricsystems obtain and compare data for physical features, such asfingerprints, voice, facial characteristics, iris patterns, handgeometry, retina patterns, and hand/palm vein structure. Differenttraits impose different constraints on the identification processes ofthese systems. For example, fingerprint recognition systems require theperson to be identified to contact an object directly for the purpose ofobtaining fingerprint data from the object. Similarly, retina patternidentification systems require a person to allow an imaging system toscan the retinal pattern within one's eye for an image capture of thepattern that identifies a person. Facial feature recognition systems,however, do not require direct contact with a person and these biometricsystems are capable of capturing identification data without thecooperation of the person to be identified.

One trait especially suited for identification is sclera patterns in aperson's eye. The human eye sclera provides a unique trait that changeslittle over a person's lifetime. It also provide multi-layer informationthat can be used for liveness test. It is important to design a methodto segment and match the sclera pattern accurately and robustly.

SUMMARY

A method has been developed that obtains an identificationcharacteristic from the sclera of a subject's eye. The method includesacquiring an image of an eye of a subject, segmenting the eye image intodifferent regions, extracting features in a sclera region segmented fromthe eye image, and generating data identifying at least one featureextracted from the sclera region of the eye image.

A system that implements the method also obtains an identificationcharacteristic from the sclera of a subject's eye. The system includes adigital camera configured to acquire an image of an eye of a subject, adigital image processor configured to segment the eye image intodifferent regions, to extract features in a sclera region segmented fromthe eye image, and to generate data identifying at least one featureextracted from the sclera region of the eye image, and a database forstorage of identifying data.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 depicts a frontal image of a human eye and identifies therelevant parts of the image;

FIG. 2 is a flow diagram of a method for identifying a subject fromfeatures in a sclera region of an eye image;

FIG. 3 is a flow diagram of a method for segmenting an image of an eye,including segmenting the sclera and iris regions;

FIG. 4 depicts a representation of the sclera region of an eye imageformed from two different representations of the sclera in the eyeimage;

FIG. 5 depicts an odd and even series of two dimensional Gabor filters;

FIG. 6 depicts a line segment being used to approximate the position andshape of a portion of a vein in the sclera;

FIG. 7 is an example template of the sclera region of an eye image withan edge portion defined around the sclera region;

FIG. 8 depicts frontal images of the sclera region of an eye, anenhanced view of the sclera veins, a binary representation of the scleraveins, and the sclera veins represented as single-pixel wide lines; and

FIG. 9 is a block diagram of a system that may be used to implement themethod of FIG. 2.

DETAILED DESCRIPTION

The method and system discussed below use patterns in the sclera of aneye, especially a human eye, for identification. Therefore, the presentinvention provides an identification technique based on the recognitionof the unique features of the sclera, referred to herein as “ScleraRecognition”. In general, the method of identification includesilluminating an eye, obtaining an image of the eye (sclera, iris, andpupil), segmenting the image, extracting features from the scleraregion, registering those extracted features, and generating a template.This template may be stored and compared to templates obtained from eyeimages of other subjects to identify the subsequent subject as being thesame subject from which the stored template was obtained.

An illustration of a human eye is shown in FIG. 1. The eye 10 includes apupil 14 surrounded by an iris 18. A limbic boundary 20 separates theiris 18 from the sclera region 22. A medial point 26 identifies the areawhere a tear duct is typically located and the lateral point 30identifies an outside edge of the image. Within the sclera 22 are bloodvessels 34 that form patterns. These patterns have been determined to besufficiently unique that may be used to identify a subject.

A method for identifying a person from an image of a person's eye isshown in FIG. 2. The process 200 begins with an acquisition of a colorimage of a subject's eye (block 204). Imaging of an eye may includeillumination of the eye in near infrared, infrared, visible,multispectral, or hyperspectral frequency light. The light may bepolarized or non-polarized and the illumination source may be close orremote from the eye. A light source close to an eye refers to a lightsource that directly illuminates the eye in the presence of the subject.A remote light source refers to a light source that illuminates the eyeat a distance that is unlikely to be detected by the subject. As notedbelow, adjustments may be made to the image to compensate for imagedeformation that may occur through angled image acquisition or eyemovement. Thus, the eye image may be a frontal image or a deformedimage. The image acquisition may be performed with a digital camerahaving an adequate resolution for imaging blood vessels within thesubject's eye.

The process of FIG. 2 continues by segmenting the image into variousregions (block 208) with an example segmentation process depicted inFIG. 3. The image may be downsampled to a smaller size before furtherprocessing occurs (block 308). In the embodiment of FIG. 3, thedownsampling produces a downsampled image 1/25^(th) the size of theoriginal image, but different amounts of downsampling may be useddepending upon the original image size and the available processinghardware.

The binary data representing pixels of the eye image are converted froman initial red, green, blue (RGB) color space, to an intermediate luma,red-difference, blue-difference (YCrCb) color space, and then into ahue, saturation, brightness value (HSV) color space (block 312) usingtransformations listed below.

$\begin{matrix}{\begin{bmatrix}Y \\C_{R} \\C_{B}\end{bmatrix} = {{\begin{bmatrix}{.299} & {.587} & {.114} \\{- {.169}} & {- {.331}} & {.499} \\{.499} & {- {.418}} & {- {.0813}}\end{bmatrix}\begin{bmatrix}R \\G \\B\end{bmatrix}} + \begin{bmatrix}0 \\128 \\128\end{bmatrix}}} & {{Equation}\mspace{14mu} 1} \\{\begin{bmatrix}H \\S \\V\end{bmatrix} = \begin{bmatrix}{\tan^{- 1}\frac{C_{B}}{C_{R}}} \\\sqrt{C_{R}^{2} + C_{B}^{2}} \\Y\end{bmatrix}} & {{Equation}\mspace{14mu} 2}\end{matrix}$

In Equation 1, the red (R), green (G), and blue (B) numeric valuescorresponding to each pixel undergo a cross product matrix transformwith the weighting matrix of equation 1. The numeric RGB values inequation 1 are in a range of 0 to 255, but larger or smaller rangesusing modified matrix coefficients are possible for alternative binaryimage formats. The resulting matrix is then adjusted by adding 128 toeach of the Cr and Cb values, resulting in a YCrCb matrix. The YCrCbmatrix is then transformed into an HSV matrix using the transformationslisted in equation 2. The transformations listed above are repeated foreach pixel in the eye image, producing an image in the HSV color space.Both the original RGB and the transformed HSV color space values areused in different steps of the segmentation process of FIG. 3.

The process of FIG. 3 continues by estimating the area of the sclerawithin the HSV image (block 316). The estimated sclera area isdetermined using a combination of two different techniques. Onetechnique is rooted in the observation that the eye image includes botha “skin” portion, including the eyelids, and a “non-skin” portioncontaining the sclera. Using color distance maps, the skin area of theimage is determined, and the inverse portions of the image are theninterpreted as being the sclera. Two example color distance map (CDM)equations are listed below.

$\begin{matrix}{{CDM}_{1} = \left\{ \begin{matrix}{1,} & \begin{matrix}\begin{matrix}{{R > 95},{G > 40},{B > 20},} \\{{{{\max\left( {R,G,B} \right)} - {\min\left( {R,G,B} \right)}} > 15},}\end{matrix} \\{{{{R - G}} > 15},{R > G},{R > B}}\end{matrix} \\{0,} & {else}\end{matrix} \right.} & {{Equation}\mspace{14mu} 3} \\{{CDM}_{2} = \left\{ \begin{matrix}{1,} & \begin{matrix}\begin{matrix}{{R > 220},{G > 210},{B > 170},} \\{{{{\max\left( {R,G,B} \right)} - {\min\left( {R,G,B} \right)}} > 15},}\end{matrix} \\{{{{R - G}} \leq 15},{R > B},{B > G}}\end{matrix} \\{0,} & {else}\end{matrix} \right.} & {{Equation}\mspace{14mu} 4} \\{{S_{1}\left( {x,y} \right)} = \left\{ \begin{matrix}{1,} & {{{{CDM}_{1}\left( {x,y} \right)}\mspace{14mu}{OR}\mspace{14mu}{{CDM}_{2}\left( {x,y} \right)}} = 0} \\{0,} & {else}\end{matrix} \right.} & {{Equation}\mspace{14mu} 5}\end{matrix}$Equation 3 describes an example CDM/for a photograph taken using anatural source of illumination, such as sunlight. Equation 4 is anexample CDM₂ for a photograph taken using a flash illuminator. Each ofthe CDMs is described using the red, green, blue (RGB) color space, withtypical values of each RGB component ranging from 0 to 255. The CDM maybe applied to each RGB pixel, yielding a 0 or 1 depending upon thepixel's color threshold value. Depending upon the illumination source,either CDM/or CDM₂ is used produce a binary sclera map S at each pixelposition x, y of the original image as show in equation 5. The twodimensional binary sclera map contains a value of 1 corresponding to apixel determined to be in the sclera, or 0 for a pixel determined to beoutside the sclera.

Another method of producing a sclera map is rooted in the observationthat the sclera is also known as the “white” portion of the eye. Usingthe HSV values image values, each pixel in the image is assigned a 0 or1 value according to the threshold equation below.

$\begin{matrix}{{S_{2}\left( {x,y} \right)} = \left\{ \begin{matrix}{1,} & \begin{matrix}\begin{matrix}{{{if}\mspace{14mu}{H\left( {x,y} \right)}} \leq {th}_{h}} \\{{{and}\mspace{14mu}{S\left( {x,y} \right)}} \leq {th}_{s}}\end{matrix} \\{{{and}\mspace{14mu}{V\left( {x,y} \right)}} \geq {th}_{v}}\end{matrix} \\{0,} & {else}\end{matrix} \right.} & {{Equation}\mspace{14mu} 6}\end{matrix}$The threshold values th_(h), th_(s), and th_(v) in Equation 6 areheuristically determined in order to set the threshold for the hue inthe approximately lower ⅓ of the image, the saturation in theapproximately lower ⅖ of the image, and the brightness in the upper ⅔ ofthe image. These heuristic thresholds are calculated based on histogramdistributions of each image using the equations listed below.th _(h)=arg{t|min|Σ_(x=1) ^(t) p _(h)(x)−T _(h)|},  Equation 7th _(s)=arg{t|min|Σ_(x=1) ^(t) p _(s)(x)−T _(s)|},  Equation 8and th _(v)=arg{t|min|Σ_(x=1) ^(t) p _(v)(x)−T _(v)|}  Equation 9

In Equations 7-9, p_(h)(x) is the normalized histogram of the hue image,p_(s)(x) is the normalized histogram of the saturation image, andp_(v)(x) is the normalized histogram of the value image. The fixedthreshold values T_(h), T_(s), and T_(v) are chosen to be ⅓, ⅖, and ⅔,respectively, matching the preferred thresholds discussed above. Theresult S₂(x,y) is the binary sclera map produced with the HSV method.

Morphological operations are applied to each binary sclera map S₁ and S₂in order to eliminate stray pixels that do not match the surroundingpixels. The preferred result of the morphological operations are twocontiguous regions in each binary sclera map corresponding to theportions of the sclera 22 on the left and right side of the iris 18 asdepicted in FIG. 1.

Continuing to refer to FIG. 3, a convex hull calculation is applied toeach binary sclera map (block 320). The convex hull is the minimalconvex set of points that contains the entire original set. It can bevisualized as the boundary of the set of points that contains all of thepoints, without requiring a concave segment, or as if an elastic bandwere stretched around the set of points. In the case of the sclera mapsS₁ and S₂, the convex hull is formed around the set of points determinedto be in the sclera region of the image.

The complete estimated sclera region is formed by selectively combiningthe binary maps S₁ and S₁ after the convex hull operation has beencompleted (block 324). The regions to be selected for the combinedbinary sclera map are chosen based on homogeneity of the pixel colorsfrom the original image that correspond to the sclera regions of eachbinary sclera map. The sclera map region that corresponds to the mosthomogeneous color distribution in the original downsampled image ischosen to represent the sclera area. The homogeneity is determined usingthe equation below.

$\begin{matrix}{{r = {\arg\left\{ {\mathbb{i}} \middle| {\min\;{\sum\limits_{{({x,y})} \in S_{i}}\left( {{I\left( {x,y} \right)} - m_{i}} \right)^{2}}} \right\}}},} & {{Equation}\mspace{14mu} 10}\end{matrix}$The preferred region r is selected from one of the two binary scleramaps based on the minimum standard deviation between image intensity ofindividual I(x, y) in the region, and the mean intensity m_(i) for allpixels in the region. The lower standard of deviation from the meanvalue indicates a more homogenous region, and that region is chosen torepresent the estimated sclera area.

An example of two binary sclera maps and a combined estimated scleraarea map 400 is depicted in FIG. 4. The first sclera image 402 is shownas an example of the binary sclera map S₁, and the second sclera image410 is an example of the binary sclera map S₂. Binary sclera map 402contains a left sclera portion 404, and right sclera portion 408, asdoes binary sclera map 410 at 412 and 416, respectively. The homogeneityof each sclera region from binary sclera maps 402 and 410 is calculatedusing equation 10 and the color values from the original downsampledimage that correspond to the locations in each sclera region. In theexample of FIG. 4, the fused estimated sclera region 418 combines theleft sclera region 404 of binary sclera map 402 with the right scleraportion 416 of binary sclera map 410. The fused sclera region 418 ofFIG. 4 is merely presented as one possible example of an estimatedsclera area, and the method of FIG. 3 produces different results fordifferent images. For example, the homogenous regions could both becontained in either S₁ or S₂.

Referring again to FIG. 3, the fused sclera region map forms the top andbottom boundaries for initiating eyelid and iris detection (block 328).Because the sclera is much lighter than the iris area, edge detectionmay be used to identify the edges of the limbic boundary between theiris and the sclera. After the initial boundaries are detected, theedges defining the sclera, iris, and eyelid regions may be furtherrefined using Fourier active contour techniques known to the art. Analternative method uses two dimensional polynomial curves to model theeyelids. Any eyelashes that extend into the sclera region of the imageare also detected and removed. These eyelash portions are modeled ashigh edge areas with a light background. The refinements removing theiris, eyelid, and eyelash elements further define the estimated portionof the downsampled image that corresponds to the sclera. An example of asegmented sclera image is depicted in FIG. 8 at 803 and 804. The iris 18is also segmented in the eye image (block 332). If the sclera is ofprimary concern, the iris may be segmented using a circular estimatedregion placed between the left and right portions of the refinedestimated sclera region.

The refined estimated binary sclera region forms an image mask pattern(block 336). This mask is then upsampled back to the size of theoriginal image (block 340). In the example of FIG. 3, the upsamplingratio would increase the original mask by a factor of 25 to reverse thedownsampling. The binary image mask is aligned with the original image,and the portions of the original image that match the binary mask arepreserved, resulting in a segmented sclera portion of the original image(block 344).

Returning to the process of FIG. 2, the segmented sclera image containsfeatures such as vein patterns, vein lines, and extrema points that areenhanced to improve feature detection (block 212). The surface of an eyeis often highly reflective, which makes focusing of an imaging devicephotographing the eye difficult. The result is that features within thesclera region may be blurred, and have a low contrast ratio in theimage. In order to enhance feature patterns, a Gabor filtering techniquemay be used to differentiate the features from the surrounding sclera.The Gabor filtering process applies the Gabor filter of equation 11using the transformation of equation 12 on the segmented sclera image.

$\begin{matrix}{{G\left( {x,y,\vartheta,s} \right)} = {{\mathbb{e}}^{- {\pi{(\frac{{({x - x_{0}})}^{2} + {({y - y_{0}})}^{2}}{s^{2}})}}}{\mathbb{e}}^{{{- 2}\pi\;{{\mathbb{i}}{({{\cos\;{\vartheta{({x - x_{0}})}}} + {\sin\;{\vartheta{({y - y_{0}})}}}})}}},}}} & {{Equation}\mspace{14mu} 11} \\{\mspace{79mu}{{I_{F}\left( {x,y,\theta,s} \right)} = {{I\left( {x,y} \right)}*{G\left( {x,y,\theta,s} \right)}}}} & {{Equation}\mspace{14mu} 12}\end{matrix}$G(x, y, θ, s) represents one or more two-dimensional Gabor filtersoriented in different directions. In the Gabor filter of Equation 11, xand y represent the center frequency of the filter, θ is the angle ofsinusoidal modulation, and s is the variance of a Gaussian function.I(x, y) is the pixel intensity at each point in the image which isconvolved with the Gabor filter. The convolution results in a filteredGabor image I_(F) (x, y, θ, s) of Equation 12 at a given orientation θand Gaussian variance s. By altering θ, the Gabor filters may be placedinto multiple orientations producing a unique filtered image functionI_(F) (x, y, θ, s) for each value of θ.

Two example Gabor filter sets are depicted in FIG. 5. The first filterset 504 is an even Gabor filter set with multiple orientations 508, 512,516, and 520. The second filter set 524 is an odd Gabor filter set withmultiple orientations 528, 532, 536, and 540. Either the even or oddfilter set may be used in the transformation of equation 11.

The multiple Gabor filtered images are fused into a vein-boosted imageF(x, y) using the following equation.F(x,y)=√{square root over (Σ_(θεθ)Σ_(sεS)(I _(F)(x,y,θ,s))²)}  Equation13

An example of a portion of the sclera image without Gabor filtering isdepicted in FIG. 8 at 804. The image includes vein features 806. Anexample Gabor filtering process using the even filter set 504 isapplied, and the individual results are fused using Equation 13 toproduce results 808 with vein structure 810.

The Gabor filtered image is converted to a binary image using anadaptive threshold, which is determined based on the distribution offiltered pixel values via the following equations.

$\begin{matrix}{{B\left( {x,y} \right)} = \left\{ \begin{matrix}{1,} & {{{F\left( {x,y} \right)} > {th}_{b}},} \\{0,} & {else}\end{matrix} \right.} & {{Equation}\mspace{14mu} 14} \\{{th}_{b} = {\arg{\left\{ \left. {{t{\min }{\sum\limits_{x = 1}^{t}{P_{edge}(x)}}} - T_{B}} \right| \right\}.}}} & {{Equation}\mspace{14mu} 15}\end{matrix}$The binary enhanced image B(x, y) has a 1 value where the fusedGabor-filtered image F(x, y) exceeds threshold th_(b), and is 0otherwise. The threshold th_(b) is calculated using p_(edge), thenormalized histogram of the non-zero elements of F(x, y). T_(b) isselected to be ⅓ in the example embodiment because the zero elements ofthe filtered image may outnumber the non-zero image elements, and thevascular patterns often have a higher magnitude than the background. Anexample of a binary enhanced image of the segmented sclera region isdepicted in FIG. 8 at 812, with vein features 814.

Referring to the example process of FIG. 2, the sclera featurescontained within the binary enhanced image are extracted using a linesegment descriptor technique (block 216). The thickness of sclerafeatures, including the thickness of veins in the sclera, variesdepending upon the dilation or constriction of the veins. Thisvariability results in veins appearing thicker or thinner inunpredictable ways, and some thinner veins disappear when constricted.Binary morphological operations known to the art reduce the detectedvein structure to a single-pixel wide skeleton, and remove the branchpoints where multiple veins overlap. The branch points are removedbecause sclera veins move relative to each other over time, leading tochanges in branch points that could produce inaccurate matching results.The morphological operations leave a set of single-pixel wide linesrepresenting the vein structure, with one example depicted in FIG. 8 at816 with single-pixel wide veins 818.

The line segment descriptor technique continues with a line parsingsequence. The line parsing sequence converts the curved linesrepresenting sclera vein features into a series of linear elements thatmay be stored in a computer database. The original vein features areapproximated by line segments, and the line segments are recursivelysplit into smaller segments until the vein features are substantiallylinear.

An example of a line segment in the parsing sequence is depicted in FIG.6. A fixed central reference point 604, usually the center of the pupil606, is used as a central point from which radial lines such as line 620extend. The use of a known central point allows for correct relativepositions of vein features to be recorded even if the eye moves to adifferent position relative to the imaging device. Each radial line 620intersects the center point of a linear element 628. The linear element628 approximates a non-linear section of a vein feature 624. The linearelement 628 is described by the polar coordinate 618 of its centerpoint, its length, and the angle 616 of the linear element 628 from ahorizontal line 622 (ø). The polar coordinate of the center of linearelement 628 is the radius of radial line 620 and the angle 624 of theradial line (θ) from a horizontal line 622. By storing the identifyingelements for each linear element 628, the vein features present in thesclera may be extracted and stored in a computer readable manner. Thecoordinates (x_(p), y_(p)) locating the pupil's center 604 are alsostored, providing an absolute position from which each linear element628 may be referenced by polar coordinates.

During eye image acquisition, the eye may move or the camera may bepositioned at an angle with respect to the eye being imaged. Thesefactors affect the location of the sclera patterns presented inside theimage. To remove these effects, sclera template registration isperformed (block 220). The location of the limbic boundary, pupilcenter, limbic center, medial, and/or lateral may be used to define alocation of features/patterns and further used for registration of thefeatures/patterns. Sclera region of interest (ROI) selection achievesglobal translation, rotation, and scaling-invariance. In addition, dueto the complex deformation that can occur in the vein patterns, theregistration scheme accounts for potential changes in vein patternswhile maintaining recognition patterns with acceptable false-positiverates.

A technique for registering the sclera template features based on randomsample consensus (RANSAC) is an iterative model-fitting method that canregister the sclera features. The RANSAC method registers thecoordinates of the center points 618 of each linear element 628 used tomodel the sclera veins. The center point coordinates and angles, but notthe lengths of the linear descriptors are registered in order to preventfalse-accepts due to over-fitting the registered features.

When an eye has been registered via the registration process in block220, an identification system may use the registered template to performmatching between the stored template and an image of a candidate eye.This process commonly occurs in biometric authentication systems where auser is registered with the identification system, and the userauthenticates with the identification system at a later time. Theregistration minimizes the minimum distance between the recorded testtemplate and the target templates that are acquired later for matchingwith the test template. This reduces artificially introduced falseaccepts because different parameters are used for registration than areused for matching, so the preferred registration and preferred matchingis different for templates that should not match. The registrationprocess randomly chooses two points—one S_(xi) from the test template,and one S_(yj) from the target template. The registration process alsorandomly chooses a scaling factor and a rotation value, based on aprioriknowledge of the template database. Using these values, a fitness valuefor the registration using these parameters is calculated for eachsegment in the image using the segment's polar coordinates r and θ andthe line segment's angle ø. The test template parameter S_(xi) andtarget template parameter S_(yj), are defined below.

$\begin{matrix}{S_{xi} = {{\begin{pmatrix}\theta_{xi} \\r_{xi} \\\varnothing_{xi}\end{pmatrix}{\mspace{11mu}\;}{and}\mspace{14mu} S_{yj}} = \begin{pmatrix}\theta_{yj} \\r_{yj} \\\varnothing_{yj}\end{pmatrix}}} & {{Equation}\mspace{14mu} 16}\end{matrix}$An offset vector is calculated using the shift offset and randomlydetermined scale and angular offset values, s₀ and θ₀. The polarcoordinates are transformed into Cartesian coordinates and combined withthe scale s₀ and angular offset θ₀ to describe the line segment φ₀ inequations 17-19.

$\begin{matrix}{\varphi_{0} = \begin{pmatrix}x_{o} \\y_{o} \\s_{o} \\\varnothing_{o}\end{pmatrix}} & {{Equation}\mspace{14mu} 17} \\{x_{o} = {{r_{xi}\cos\;\theta_{xi}} - {r_{yj}\cos\;\theta_{yj}}}} & {{Equation}\mspace{14mu} 18} \\{y_{o} = {{r_{xi}\sin\;\theta_{xi}} - {r_{yj}\sin\;\theta_{yj}}}} & {{Equation}\mspace{14mu} 19}\end{matrix}$The fitness of two descriptors is the minimal summed pairwise distancebetween the two descriptors S_(x) and S_(y) given offset vector, φ₀.

$\begin{matrix}{{D\left( {S_{x},S_{y}} \right)} = {\underset{\varphi_{0}}{\arg\;\min}{\overset{\sim}{D}\left( {S_{x},S_{y},\varphi_{0}} \right)}}} & {{Equation}\mspace{14mu} 20} \\{{\overset{\sim}{D}\left( {S_{x},S_{y},\varphi_{0}} \right)} = {\sum\limits_{x_{i} \in {Test}}{\min\;{{Dist}\left( {{f\left( {S_{xi},\varphi_{0}} \right)},S_{y}} \right)}}}} & {{Equation}\mspace{14mu} 21}\end{matrix}$Where f(S_(xi),φ₀) is the function that applies the registration giventhe offset vector to a sclera line descriptor.

$\begin{matrix}{{f\left( {S_{xi},\varphi_{0}} \right)} = \begin{pmatrix}{\cos^{- 1}\left( \frac{r_{{{xi}\;\cos\;\theta_{xi}} + x_{o}}}{s_{o}r_{xi}} \right)} \\\frac{r_{{{xicos}\;\theta_{xi}} + x_{o}}}{\cos\left( {\theta_{xi} + \varnothing_{o}} \right)} \\\varnothing_{xi}\end{pmatrix}} & {{Equation}\mspace{14mu} 22}\end{matrix}$The minimum pairwise distance between the selected test template linesegment S_(xi) and the entire set of target template line segments Sy iscalculated using the equation listed below. The line segment in thetarget template closest to segment S_(xi) is assumed to be the nearestmatching sclera vein segment. This calculation allows for matching whenthe sclera veins in the target image have moved after the test templatewas recorded.minDist(S _(xi) ,S _(y))=argmin_(j) {d(S _(xi) ,S _(yj))}  Equation 23With the distance between two points calculated using the equationlisted below.d(S _(xi) ,S _(yj))=√{square root over ((x _(o))²+(y _(o))²)}{squareroot over ((x _(o))²+(y _(o))²)}  Equation 24

Where, Test is the set of descriptors in the stored test template,Target is the set of descriptors in the newly acquired target template,S_(xi) is the first descriptor used for registration, S_(yj) is thesecond descriptor, φ₀ is the set of offset parameter values, f(S_(xi),φ₀) is a function that modifies the descriptor with the given offsetvalues, S is the scaling factor, and is the rotation value which isdetermined by the sclera image resolution and system application. Theprocess performs some number of iterations, recording the values φ₀ thatare minimal in D(S_(x), S_(y)).

A template for future comparisons is generated from the extractedfeatures (block 220). The template may be, for example, a bit map of thesclera region that identifies the features, a list of features withpositional information about each feature, or a set of descriptors forthe extracted features. To generate the template, the location of theregistered extrema points or a set of descriptors for the features aresaved. Descriptors refer to the model parameters for the detectedfeature edges that were obtained through known curve fitting techniques,wavelets, neural network, filtering methods, and other patternrecognition methods. The parameters of the fitted curves are saved asdescriptors for the extrema points. The template may be represented in abinary, integer, or floating number format. The template may now be usedto identify another image of an eye as corresponding to or notcorresponding to the template. The template generation process describedabove is merely illustrative of an appropriate process for modeling animage of the eye. Alternative processes include using a set of area,line, and/or point descriptors; a set of wavelet co-efficiencies,magnitudes, phases, or a combination thereof; and a set of vectorsand/or matrices.

An identification or matching process (block 224, FIG. 2) may now beperformed by repeating the eye image processing (block 204 to 220) toobtain another eye image and generate a template for the image, and thencompare the generated template for the second image to a stored templateto determine whether the two templates correspond to one another. Thetemplates used in the comparison may not be completely accurate. Forexample, the heuristics used in segmenting the sclera may result ininaccuracies in the edge areas of the segmented sclera. As depicted inFIG. 7, each sclera template is weighted to discount the areas 794 nearthe edge of each template region where the inaccuracies occur. Thecentral portion of the sclera 708 is weighted more heavily.

The matching process compares the line segments S_(i) in the storedtemplate with line segments S_(j) stored in the template generated fromthe second eye image. The matching process produces a match scorem(S_(i), S_(j)) for each line segment in the stored template using theequation below.

$\begin{matrix}{{m\left( {S_{i}S_{j}} \right)} = \left\{ \begin{matrix}{{{w\left( S_{i} \right)}{w\left( S_{j} \right)}},} & \begin{matrix}{{d\left( {S_{i},S_{j}} \right)} \leq D_{match}} \\{and}\end{matrix} \\{0,} & \begin{matrix}{{{\varnothing_{i} - \varnothing_{j}}} \leq \varnothing_{match}} \\{else}\end{matrix}\end{matrix} \right.} & {{Equation}\mspace{14mu} 25}\end{matrix}$In Equation 25, d(S_(i), S_(j)) is the Euclidian distance between thecenter points of S_(i) and S_(j) calculated in Equation 23. D_(match) isa predetermined threshold value for the distance between line segmentsconsidered to match, and ø_(match) represents the differences betweenthe angles of line segments considered to match. In the exampleembodiment of FIG. 2, D_(match) is determined to be 5 pixels, andø_(match) is determined to be a 10° angle. If a line segment S_(i) iswithin D_(match) pixels and at an angle within ø_(match) of the linesegment S_(j), then the match results m(S_(i), S_(j)) are updated withthe weighted values w(S_(i)) and w(S_(j)) of each line segmentmultiplied together. The example weighted values are either 1, 0.5, or 0depending upon whether the line segment is located in the fully weightedsclera portion 708, the edge of the sclera region 704, or outside thesclera 712 respectively. If either the distance or the angle of S_(i)and S_(j) are greater than the threshold values D_(match) and ø_(match),no match is found and m(S_(i), S_(j)) is 0.

The matching scores m(Si, Sj) for individual line segments are summed toproduce an overall matching score M using the equation below.

$\begin{matrix}{M = \frac{\sum_{{({i,j})} \in {Matches}}{m\left( {S_{i},S_{j}} \right)}}{\min\left( {{\sum_{i \in {Target}}{w\left( S_{i} \right)}},{\sum_{j \in {Test}}{w\left( S_{j} \right)}}} \right)}} & {{Equation}\mspace{14mu} 26}\end{matrix}$Matches is the set of all line segments that matched with non 0 values,Test is the stored template that is used to test the new template Targetto determine if the sclera images match. If the sum M exceeds apredetermined threshold, then the new image is considered to match thestored template, otherwise the two images are not considered matches. Inan embodiment of a biometric identification system using the process ofFIG. 2, a match occurs when a person with a registered sclera templateis authenticated with the identification system using the processdepicted in FIG. 2.

A system that may be used to implement the image processing methoddescribed above is shown in FIG. 9. The system 900 includes a digitalcamera 904, a digital image processor 908, and a database 912. Thedigital camera, which may generate color or grayscale images, is locatedat a position where a subject may be standing or seated. The camera neednot be positioned where the subject is aware of the camera and, thus,cooperative for providing an image of the subject's eye. The camera maybe, for example, a Sony DSLR-A100 camera, although any high resolutiondigital camera is appropriate. Other cameras that may be used includenear infrared, infrared, visible, multispectral, and hyperspectralcameras. The digital image processor 908 may be a general purposemicroprocessor or a special purpose digital signal processor (DSP). Theprocessor is provided with appropriate interface circuitry forretrieving the image signal from the camera or a stored template fromthe database 912. Instructions to be executed by the processor 908 maybe stored in an internal memory of the processor or in an externalmemory coupled to the processor in a known manner. Execution of thestored instructions by the processor 908 results in the system 900performing an image processing method similar to the one described abovewith reference to FIG. 2. A template generated by the processor 908 maybe stored in the database 912 for future comparisons or the generatedtemplate may be compared to a stored template to determine whether asubject being imaged corresponds to a subject having a template storedin the database 912. Database 912 may be any appropriate data managementsystem and storage media, such as a database management system having ahard disk drive or other non-volatile memory. The templates stored inthe database 912 may be indexed with reference to extrema points ortypes of fitted curves.

In another embodiment, a system and method may use structure within aniris as well as patterns within a sclera to improve recognition accuracyand increase the degree of the freedom in subject recognition. The useof iris and sclera features for subject identification is referred toherein as “iris and sclera multimodal recognition”. A method forimplementing iris and sclera multimodal recognition includes eyeillumination, eye image acquisition (sclera, iris, and pupil), imagesegmentation, feature extraction, feature registration, and templategeneration. This method is similar to the one described above except theimage segmentation retains the iris in the acquired image, the featureextraction includes structure within the iris, and the templategeneration identifies sclera patterns and iris structure. The featureextraction of iris could be wavelet based method, descriptor basedmethod, and/or spatial-domain method. As described above, a storedtemplate may be compare to a template obtained from another acquired eyeimage to identify the subject in the second image as corresponding to ornot corresponding to the subject for the stored template.

The comparison of templates in the iris and sclera multimodal system mayuse feature level fusion, template level fusion, and/or score levelfusion. For example, the sclera and iris regions may be processedseparately and the templates generated for each separate region may thenbe stored for later comparison. Templates generated from a lateracquired image for both the iris and sclera areas may be separatelycompared to one or more stored templates to generate a pair of matchingscores. If both matching scores are higher than the matching thresholds,the subjects are deemed the same. If one of the scores does not meet orexceed the matching threshold, the context of the recognition scenariomay be used to determine the criteria for a match. For example, in ahighly secured situation, one low matching score may be sufficient toevaluate a subject as not corresponding to the subject for a storedtemplate. In a less secured scenario, such as access to a home computer,one matching score exceeding one threshold by a predetermined percentagemay be adequate to declare the subjects as corresponding to one another.In a similar manner, sclera recognition may be combined with facerecognition, skin tissue recognition, or some other biometriccharacteristic recognition system to improve recognition accuracy forthe system.

Those skilled in the art will recognize that numerous modifications canbe made to the specific implementations described above. Therefore, thefollowing claims are not to be limited to the specific embodimentsillustrated and described above. The claims, as originally presented andas they may be amended, encompass variations, alternatives,modifications, improvements, equivalents, and substantial equivalents ofthe embodiments and teachings disclosed herein, including those that arepresently unforeseen or unappreciated, and that, for example, may arisefrom applicants/patentees and others.

What is claimed is:
 1. A method for obtaining an identification characteristic for a subject comprising: acquiring an image of an eye of a subject with a digital camera; segmenting the eye image into different regions with a digital image processor; extracting features in a sclera region segmented from the eye image with the digital image processor, the extraction comprising: generating with the digital image processor a linear representation of at least one vein in the sclera region segmented from the eye image, the linear representation including a plurality of line segments that correspond to the at least one vein; generating data identifying at least one feature extracted from the sclera region of the eye image with the digital image processor, the generation comprising: identifying with the digital image processor data corresponding to a center and angle of each line segment in the plurality of line segments with reference to a center of a pupil in the eye image; and storing the data identifying the at least one feature in a database with the digital image processor.
 2. The method of claim 1, the eye image segmentation further comprising: segmenting the eye image into an iris region and sclera region with the digital image processor; and identifying a limbic boundary between the iris region and the sclera region with the digital image processor.
 3. The method of claim 2 further comprising: identifying a medial point position and a lateral point position in the eye image with the digital image processor.
 4. The method of claim 1 wherein the extracted feature is a blood vessel pattern in the sclera region.
 5. The method of claim 1 further comprising: generating with the digital image processor a model of eyelids in the eye image with a two dimensional polynomial curve.
 6. The method of claim 1, the data generation further comprising: identifying with the digital image processor locations for vein patterns, vein lines, and extrema points in the sclera region.
 7. The method of claim 1, the data generation further comprising: generating with the digital image processor model parameters from features via wavelet, curve fitting method, neural network, filtering method, and other pattern recognition methods.
 8. The method of claim 1 further comprising: comparing with the digital image processor identifying data generated from a first eye image to identifying data obtained from a second eye image to generate a comparison score; and determining with the digital image processor that a subject from which the first eye image is obtained corresponds to the a subject from which the second eye image is obtained in response to the comparison score being equal to or greater than a matching threshold.
 9. The method of claim 2 further comprising: extracting features in the iris region with the digital image processor; generating with the digital image processor data identifying at least one feature extracted from the iris region.
 10. The method of claim 9 further comprising: comparing identifying data generated for a feature extracted from the sclera region in a first eye image to identifying data generated for a feature extracted from the sclera region in a second eye image to generate a sclera comparison score with the digital image processor; comparing identifying data generated for a feature extracted from the iris region in the first eye image to identifying data generated for a feature extracted from the iris region in the second eye image to generate an iris comparison score with the digital image processor; and determining with the digital image processor that a subject from which the first eye image is obtained corresponds to a subject from which the second eye image is obtained in response to at least one of the sclera comparison score and the iris comparison score being equal to or greater than a matching threshold.
 11. The method of claim 1, the eye image acquisition further comprising: illuminating the eye with a light source configured to emit one of a near infrared, infrared, visible, multispectral, and a hyperspectral frequency light.
 12. The method of claim 1, the eye image acquisition further comprising: illuminating the eye with a light source configured to emit one of a polarized or non-polarized light.
 13. The method of claim 1, the eye image acquisition further comprising: illuminating the eye with a light source that is close to or remote from the eye.
 14. The method of claim 1 wherein the feature extraction is preceded by a feature enhancement method including: generating a plurality of filtered images from at least one of the segmented regions by applying a plurality of image filters with the digital image processor; and combining the filtered images into a single enhanced image of the at least one of the segmented regions with the digital image processor.
 15. The method of claim 1, the generation of identifying data further comprising: generating with the digital image processor a bit map of the sclera region that identifies features.
 16. The method of claim 1, the generation of identifying data further comprising: generating with the digital image processor a list of features with positional information about each feature.
 17. The method of claim 1, the generation of identifying data further comprising: generating with the digital image processor a template for the extracted features.
 18. The method of claim 1 wherein the digital image processor generates identifying data in one of a binary, integer, or floating number format.
 19. The method of claim 10 wherein the subject determination includes iris and sclera feature fusion.
 20. The method of claim 10 wherein the subject determination includes iris and sclera template fusion.
 21. The method of claim 10 wherein the subject determination includes iris and sclera comparison score fusion.
 22. A system for obtaining an identification characteristic for a subject comprising: a digital camera configured to acquire an image of an eye of a subject; a digital image processor configured to: segment the eye image into different regions; extract features in a sclera region segmented from the eye image the extraction comprising: generation of a linear representation of at least one vein in the sclera region segmented from the eye image, the linear representation including a plurality of line segments that correspond to the at least one vein; and generate data identifying at least one feature extracted from the sclera region of the eye image, the generation comprising: identification of data corresponding to a center and angle of each line segment in the plurality of line segments with reference to a center of a pupil in the eye image; and a database for storage of identifying data.
 23. The system of claim 22, the digital image processor being further configured to compare identifying data stored in the database to identifying data obtained from a second eye image to generate a comparison score; and determining a subject from which the stored identifying data is obtained corresponds to a subject from which the second eye image is obtained in response to the comparison score being equal to or greater than a matching threshold.
 24. The system of claim 22 wherein the digital camera is one of a near infrared, infrared, visible, multispectral, and hyperspectral camera.
 25. The system of claim 22 wherein the digital camera is one of a polarized and non-polarized camera.
 26. The system of claim 22 wherein the digital camera is located closely or remotely from the eye being imaged. 